Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add From/To options for Forward and Backward #518

Merged
merged 4 commits into from
Jul 20, 2014
Merged

Conversation

longjon
Copy link
Contributor

@longjon longjon commented Jun 19, 2014

This PR

  • adds six methods to Net for starting or stopping the forward or backward passes at particular layers, and
  • exposes those methods in pycaffe via keyword args to Net.forward and Net.backward.

This is useful

  • to save computation time when one only wants lower-layer features, without hacking up the prototxt file, and
  • to run experiments that involve partial forward, partial backprop, or even changing intermediate data in the middle of a forward or backward pass.

@shelhamer
Copy link
Member

Nice!

@briancheung you might like to have this for some experiments. I remember
you were interested in occlusion studies.

Le jeudi 19 juin 2014, longjon notifications@github.com a écrit :

This PR

  • adds six methods to Net for starting or stopping the forward or
    backward passes at particular layers, and
  • exposes those methods in pycaffe via keyword args to Net.forward and
    Net.backward.

This is useful

  • to save computation time when one only wants lower-layer features,
    without hacking up the prototxt file, and
  • to run experiments that involve partial forward, partial backprop,
    or even changing intermediate data in the middle of a forward or backward
    pass.

You can merge this Pull Request by running

git pull https://github.com/longjon/caffe fromto

Or view, comment on, or merge it at:

#518
Commit Summary

  • add Net::Forward/Backward From/To
  • pycaffe: expose Forward/Backward From/To as kwargs start and end

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#518.

Evan Shelhamer

@briancheung
Copy link

Great! This will clean things up a bit, is there documentation for the
Python wrappers?

On Wed, Jun 18, 2014 at 9:57 PM, Evan Shelhamer notifications@github.com
wrote:

Nice!

@briancheung you might like to have this for some experiments. I remember
you were interested in occlusion studies.

Le jeudi 19 juin 2014, longjon notifications@github.com a écrit :

This PR

  • adds six methods to Net for starting or stopping the forward or
    backward passes at particular layers, and
  • exposes those methods in pycaffe via keyword args to Net.forward and
    Net.backward.

This is useful

  • to save computation time when one only wants lower-layer features,
    without hacking up the prototxt file, and
  • to run experiments that involve partial forward, partial backprop,
    or even changing intermediate data in the middle of a forward or
    backward
    pass.

You can merge this Pull Request by running

git pull https://github.com/longjon/caffe fromto

Or view, comment on, or merge it at:

#518
Commit Summary

  • add Net::Forward/Backward From/To
  • pycaffe: expose Forward/Backward From/To as kwargs start and end

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#518.

Evan Shelhamer


Reply to this email directly or view it on GitHub
#518 (comment).

@longjon
Copy link
Contributor Author

longjon commented Jun 20, 2014

@briancheung the code has docstrings, and there are example IPython notebooks; that's the extent of it for now.

@shelhamer
Copy link
Member

How about a simple test? For instance you could run Forward() and then compare the data with a ForwardTo() and ForwardFrom() done together and do likewise for Backward().

No strong feelings on this -- if you don't think it's worth it we can just merge.

@shelhamer
Copy link
Member

It could be worth a little inline documentation to note this is not guaranteed to be correct for DAGs. The sequence of indices between the start and end is not necessarily the path between them since it is up to the order in the model definition. If this sequence is not the path, the computation should be correct but it will compute unnecessary layers off of the path. Another problem case is calling a partial forward from/to/through a fan-in on the DAG; there is no guarantee the other paths will have done their computations so that the bottoms are correct.

I'm totally fine with this, since a judicious series of partial forwards gives the correct result, and this is an expert feature. However I'd vote for documenting this. (We could even just write this all up nicely in a comment and put the link in the code.)

@shelhamer
Copy link
Member

@longjon any thoughts on test or inline documentation? If you think this is fine as is, let's merge.

@longjon
Copy link
Contributor Author

longjon commented Jul 5, 2014

@shelhamer, I agree with both your comments. Although I think the code is simple enough that I'm confident in its correctness without a test, future changes might complicate this functionality and make that no longer true. I did have to stop and think about DAGs before doing this, so it's probably worth a comment.

I'll probably be able to make those changes by the end of the coming week. I'm fine with either waiting until then or merging this now and doing those things in another PR.

@longjon
Copy link
Contributor Author

longjon commented Jul 18, 2014

@shelhamer, I've made the additions you've suggested. Everything passes for me, if Travis agrees I think this is ready for merge.

@shelhamer
Copy link
Member

Merge it if Travis passes, I am traveling today. Thanks Jon!

Le samedi 19 juillet 2014, longjon notifications@github.com a écrit :

@shelhamer https://github.com/shelhamer, I've made the additions you've
suggested. Everything passes for me, if Travis agrees I think this is ready
for merge.


Reply to this email directly or view it on GitHub
#518 (comment).

@longjon longjon merged commit 1c23286 into BVLC:dev Jul 20, 2014
longjon added a commit that referenced this pull request Jul 20, 2014
Add From/To options for Forward and Backward
@shelhamer shelhamer mentioned this pull request Aug 8, 2014
mitmul pushed a commit to mitmul/caffe that referenced this pull request Sep 30, 2014
Add From/To options for Forward and Backward
RazvanRanca pushed a commit to RazvanRanca/caffe that referenced this pull request Nov 4, 2014
Add From/To options for Forward and Backward
@longjon longjon deleted the fromto branch December 30, 2014 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants